[SPARK-7586][ML][doc] Add docs of Word2Vec in ml package #6181

yinxusen · 2015-05-15T08:35:35Z

CC @jkbradley.

JIRA issue.

AmplabJenkins · 2015-05-15T08:37:11Z

Merged build triggered.

AmplabJenkins · 2015-05-15T08:37:21Z

Merged build started.

SparkQA · 2015-05-15T08:39:24Z

Test build #32804 has started for PR 6181 at commit 1c3f389.

SparkQA · 2015-05-15T08:39:30Z

Test build #32804 has finished for PR 6181 at commit 1c3f389.

This patch fails RAT tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- public class JavaWord2VecSuite

AmplabJenkins · 2015-05-15T08:39:31Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-15T08:39:31Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32804/
Test FAILed.

SparkQA · 2015-05-16T01:06:32Z

Test build #812 has started for PR 6181 at commit 1c3f389.

SparkQA · 2015-05-16T01:06:47Z

Test build #812 has finished for PR 6181 at commit 1c3f389.

This patch fails RAT tests.
This patch merges cleanly.
This patch adds no public classes.

jkbradley · 2015-05-16T02:40:34Z

docs/ml-features.md

+{% highlight scala %}
+import org.apache.spark.ml.feature.Word2Vec
+
+val documentDF = sqlContext.createDataFrame(Seq(


Add comment in line above:

Input data: Each row is a bag of words from a sentence or document.

(Please add to other examples too.)

jkbradley · 2015-05-16T02:40:52Z

@yinxusen I tried the Scala example but got this error:

scala> val documentDF = sqlContext.createDataFrame(Seq(
     |   "Hi I heard about Spark".split(" "),
     |   "I wish Java could use case classes".split(" "),
     |   "Logistic regression models are neat".split(" ")
     | )).map(Tuple1.apply).toDF("text")
<console>:20: error: overloaded method value createDataFrame with alternatives:
  [A <: Product](data: Seq[A])(implicit evidence$4: reflect.runtime.universe.TypeTag[A])org.apache.spark.sql.DataFrame <and>
  [A <: Product](rdd: org.apache.spark.rdd.RDD[A])(implicit evidence$3: reflect.runtime.universe.TypeTag[A])org.apache.spark.sql.DataFrame
 cannot be applied to (Seq[Array[String]])
       val documentDF = sqlContext.createDataFrame(Seq(

Did these all run for you?

The Java file needs the Apache license at the top.

AmplabJenkins · 2015-05-17T11:17:10Z

Merged build triggered.

AmplabJenkins · 2015-05-17T11:17:17Z

Merged build started.

SparkQA · 2015-05-17T11:19:08Z

Test build #32935 has started for PR 6181 at commit 57a4c07.

SparkQA · 2015-05-17T11:19:14Z

Test build #32935 has finished for PR 6181 at commit 57a4c07.

This patch fails RAT tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- public class JavaWord2VecSuite

AmplabJenkins · 2015-05-17T11:19:15Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-17T11:19:15Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32935/
Test FAILed.

jkbradley · 2015-05-18T16:48:00Z

@yinxusen This needs the Apache license at the top of all files. (That is causing the RAT test failure.)

jkbradley · 2015-05-18T19:29:28Z

@yinxusen Except for the RAT test issue, this LGTM. You may encounter merge issues from other updates to ml-features.md, but they should be easy to fix.

yinxusen · 2015-05-19T03:14:52Z

@jkbradley Sorry for forgetting the Apache license again... fix it now.

yinxusen · 2015-05-19T03:20:27Z

@jkbradley Seems it has no merging problem, for now.

AmplabJenkins · 2015-05-19T03:22:10Z

Merged build triggered.

AmplabJenkins · 2015-05-19T03:22:19Z

Merged build started.

SparkQA · 2015-05-19T03:22:47Z

Test build #33050 has started for PR 6181 at commit 77014c5.

SparkQA · 2015-05-19T03:32:34Z

Test build #33050 has finished for PR 6181 at commit 77014c5.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- public class JavaWord2VecSuite

AmplabJenkins · 2015-05-19T03:32:37Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-19T03:32:38Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33050/
Test FAILed.

SparkQA · 2015-05-19T07:04:57Z

Test build #830 has started for PR 6181 at commit 77014c5.

yinxusen · 2015-05-19T07:40:33Z

@jkbradley Are there more concrete error infos for Mima error? I search from the console output of the test, but cannot locate the error.

SparkQA · 2015-05-19T09:34:58Z

Test build #830 timed out for PR 6181 at commit 77014c5 after a configured wait of 150m.

jkbradley · 2015-05-19T17:43:25Z

@yinxusen It's not actually a MIMA error; that's a bug in the script reporting on the tests. It looks to me like Jenkins being flaky.

SparkQA · 2015-05-19T17:43:47Z

Test build #831 has started for PR 6181 at commit 77014c5.

SparkQA · 2015-05-19T19:31:59Z

Test build #831 has finished for PR 6181 at commit 77014c5.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- public class JavaWord2VecSuite

jkbradley · 2015-05-19T20:43:08Z

LGTM merging into master and branch-1.4
@yinxusen Thank you!

CC jkbradley. JIRA [issue](https://issues.apache.org/jira/browse/SPARK-7586). Author: Xusen Yin <[email protected]> Closes #6181 from yinxusen/SPARK-7586 and squashes the following commits: 77014c5 [Xusen Yin] comment fix 57a4c07 [Xusen Yin] small fix for docs 1178c8f [Xusen Yin] remove the correctness check in java suite 1c3f389 [Xusen Yin] delete sbt commit 1af152b [Xusen Yin] check python example code 1b5369e [Xusen Yin] add docs of word2vec (cherry picked from commit 68fb2a4) Signed-off-by: Joseph K. Bradley <[email protected]>

CC jkbradley. JIRA [issue](https://issues.apache.org/jira/browse/SPARK-7586). Author: Xusen Yin <[email protected]> Closes apache#6181 from yinxusen/SPARK-7586 and squashes the following commits: 77014c5 [Xusen Yin] comment fix 57a4c07 [Xusen Yin] small fix for docs 1178c8f [Xusen Yin] remove the correctness check in java suite 1c3f389 [Xusen Yin] delete sbt commit 1af152b [Xusen Yin] check python example code 1b5369e [Xusen Yin] add docs of word2vec

yinxusen added 3 commits May 16, 2015 00:25

add docs of word2vec

1b5369e

check python example code

1af152b

delete sbt commit

1c3f389

jkbradley reviewed May 16, 2015
View reviewed changes

yinxusen added 2 commits May 17, 2015 19:06

remove the correctness check in java suite

1178c8f

small fix for docs

57a4c07

comment fix

77014c5

asfgit closed this in 68fb2a4 May 19, 2015

[SPARK-7586][ML][doc] Add docs of Word2Vec in ml package #6181

[SPARK-7586][ML][doc] Add docs of Word2Vec in ml package #6181

Uh oh!

Conversation

yinxusen commented May 15, 2015

Uh oh!

AmplabJenkins commented May 15, 2015

Uh oh!

AmplabJenkins commented May 15, 2015

Uh oh!

SparkQA commented May 15, 2015

Uh oh!

SparkQA commented May 15, 2015

Uh oh!

AmplabJenkins commented May 15, 2015

Uh oh!

AmplabJenkins commented May 15, 2015

Uh oh!

SparkQA commented May 16, 2015

Uh oh!

SparkQA commented May 16, 2015

Uh oh!

jkbradley May 16, 2015

Choose a reason for hiding this comment

Uh oh!

jkbradley commented May 16, 2015

Uh oh!

AmplabJenkins commented May 17, 2015

Uh oh!

AmplabJenkins commented May 17, 2015

Uh oh!

SparkQA commented May 17, 2015

Uh oh!

SparkQA commented May 17, 2015

Uh oh!

AmplabJenkins commented May 17, 2015

Uh oh!

AmplabJenkins commented May 17, 2015

Uh oh!

jkbradley commented May 18, 2015

Uh oh!

jkbradley commented May 18, 2015

Uh oh!

yinxusen commented May 19, 2015

Uh oh!

yinxusen commented May 19, 2015

Uh oh!

AmplabJenkins commented May 19, 2015

Uh oh!

AmplabJenkins commented May 19, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

AmplabJenkins commented May 19, 2015

Uh oh!

AmplabJenkins commented May 19, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

yinxusen commented May 19, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

jkbradley commented May 19, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

SparkQA commented May 19, 2015

Uh oh!

jkbradley commented May 19, 2015

Uh oh!

Uh oh!